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Abstract. Hidden Markov Models (HMMs) have become very popular as a computational tool for 
the analysis of sequential data. They are memoryless machines which transition from one internal 
state to another, while producing symbols. These symbols constitute the output of the machine and 
form an infinite time series. Analogously, Hidden Quantum Markov Models (HQMM) produce an 
infinite time series, while progressing from one quantum state to another through stochastic quantum 
operations. Here we compare 1-bit HMMs and 1 -qubit HQMMs and show that the latter can produce 
stronger correlations, although both machines are, in principle, comparable in resources. 
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INTRODUCTION 

Hidden Markov Models (HMMs) are useful computational tools for the simulation of 
stochastic processes [1, 2, 3]. They are stochastic finite-state generators which can be 
used to model a wide range of situations. Applications include speech recognition, im- 
age analysis, and the modelling of biological systems. This paper concerns their quan- 
tum mechanical counterparts [4, 5, 6]. We especially discuss Hidden Quantum Markov 
Models (HQMMs) which have recently been introduced by Monras et al. [5]. Analo- 
gously to HMMs, we expect HQMMs to find a wide range of applications, including 
the simulation of quantum systems and the modelling of biological processes [7]. As 
shown in Ref. [5], HQMMs naturally encompass HMMs in the sense that they are able 
to simulate the same and even more complex output sequences with the same number of 
internal states. 

In the next section, we present a widely used and relatively intuitive definition of 
HMMs. Afterwards, we introduce HQMMs by following the ideas of Ref. [5]. For sim- 
plicity, we focus our attention on 1-bit and 1 -qubit versions of these machines and as- 
sume output alphabets which contain only two symbols. These specific machines can 
be characterised relatively easily by a countable set of parameters which allows us to 
study their properties analytically and numerically. Calculating for example the maxi- 
mum probabilities for certain words to occur, one can show that 1 -qubit HQMMs are 
able to produce stronger correlations than 1-bit HMMs, although requiring, in principle, 
a similar amount of resources. 
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FIGURE 1. Schematic view of the time evolution of a) HMMs and b) HQMMs. 



HIDDEN MARKOV MODELS 

Hidden Markov Models (HMMs) are machines which transition from one state of a finite 
set 5? to another state, while producing a symbol of an output alphabet srf [1, 2, 3]. 
Although the actual state Y e 5? of the machine remains hidden, it fully determines the 
probability for the system to progress to a certain state X E ,Y and to generate a certain 
output symbol i e in the next time step (cf. Fig. l.a). Different definitions of HMMs 
can be found in the literature. The HMMs which we consider here are known as Mealy 
(or transition-emitting) HMMs [2]. Mealy HMMs encompass Moore (or state-emitting) 
HMMs, i.e. they contain them as a subset [3]. 

For simplicity, we consider in the following 1-bit versions of transition-emitting 
HMMs. These produce either a or a 1, while evolving between two states A and B. 
Concrete versions of 1-bit HMMs are fully characterised by eight conditional probabili- 
ties P(XJ\Y). Given these probabilities, the probability to obtain the output symbol i at 
t+ 1 equals P t+ i (i) = Y,x=a,bY<y=a,bP(Xi z I^) Pt(Y), if the machine is in state Y at time 
t with probability P t (Y). Analogously, one can show that the probability to transition 
to X at t+ 1 equals P t+ i (X) = £(=o,l Y,y=a.bP(Xi z I^) Pt(Y). Introducing the stochastic 
matrices To and T\ such that 

f P(A,i\A) P(A,i\B) \ m 
ll ~ \ P{B,i\A) P{B,i\B) ) ' UJ 

the above probability P t+ i(X) can be written in a more compact way. Combining all 
of the above equations, one can show that the probability to find for example the word 
10001 with the first letter being produced at t + 1 equals 



t+l,...,t+5 



(10001) = ( 1 1 ) T^oToToTi ( ) , (2) 



if the machine is at time t with probability P t (Y) in Y. Moreover, one can show that the 
machine is at t+ 1 in the statistical mixture described by 

p t+ m ) - ^ +T ^{p t (B) J' (3) 

when the respective output symbols are disregarded. 

In summary, a Mealy HMM is a stochastic finite-state generator [4] which is char- 
acterised by a tuple {y,£/,T}, where y denotes a finite set of states, stf = {/} is a 
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countable alphabet of output symbols, while T = {7} : i e is a set of square sub- 
stochastic matrices of the same dimension as [4]. The machine undergoes transitions 
from one internal state to another. Each transition has an associated output symbol. The 
matrices 7} summarise the probabilities for certain transitions to occur and fully charac- 
terise the machine. As illustrated in Eq. (3), the sum T = Y*iesrf Ti is a stochastic matrix 
which describes a Markovian evolution of the state space, when the output symbols are 
disregarded. 

HIDDEN QUANTUM MARKOV MODELS 

Let us now have a closer look at possible quantum versions of HMMs. For example, a 
quantum finite-state generator [4] which produces a similar output of O's and l's as a 
1-bit HMM consists of a single qubit. The qubit evolves in time according to a unitary 
operator U and undergoes projective measurements. A straightforward generalisation 
of a quantum finite-state generator is a 1 -qubit machine which experiences generalised 
quantum measurements [8]. These are, for example, described by two Kraus operators 
Kq and K\ with 

K*Ko + K}Ki = 1. (4) 

Suppose the qubit is at time t in a pure state \ \\f t ). Then it changes at t + 1 into | Vf+i) = 
^'IVfVII ' ||> if the output symbol i e {0,1} is detected. The corresponding output 
probability is ||^-|v^)|| . One way of implementing Kq and in the laboratory is to 
entangle the qubit with an ancilla qubit. The ancilla subsequently undergoes a projective 
measurement which produces either a or a 1. Before the next time step, the ancilla 
should be resetted, if we want to realise a memory-less machines. 

In order to obtain 1 -qubit machines which cannot be outperformed by 1-bit HMMs, 
we need to consider the most general transformation which a qubit can undergo [5]. The 
general definition of Hidden Quantum Markov Models (HQMMs) by Monras et al. [5] 
hence involves complete sets of Kraus operators Jf. Each HQMM is characterised by 
a set {Jtfjg/ ,Jf}, where J$? is the Hilbert space of a quantum system, s4 is again a 
countable alphabet of output symbols, and J?T = {K™ : i e 1 > m > Mi} is a complete 
set of Kraus operators [8] such that 

Mi 

£ Y^KfKT = 1. (5) 

m=\ 

In each time step, an output symbol i e srf is generated which depends only on the input 
state p t of the machine. If the output symbol i e is recorded, then the machine tran- 
sitions into Em=i K^PtK™ /Tx{-). The corresponding output probability P t +\_(i) equals 
L m Li Tr(K™p t K™*). If the output is ignored, the density matrix of the machine becomes 
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at t + 1, as illustrated in Fig. Lb. For example, one way of realising a 1-qubit HQMM is 
to select a set of Kraus operators {^0,^1} which fulfills Eq. (4) with a certain classical 
probability and to apply the corresponding Kraus operation to the qubit. This means, 
HQMMs have been defined such that they can exploit quantum coherences as well as 
classical stochastic processes to generate relatively complex time series. 

COMPARISON 

The time evolution of HMMs and HQMMs is Markovian. This means, the states and 
output symbols of these machines depend only on the respective state of the machine in 
the previous time step. However, a 1-qubit HQMM can be prepared in a huge variety of 
states, while a 1-bitHMMis always either in A or B. The state of a 1-qubit HQMM hence 
depends in general in a more complex way on the history of the machine. We therefore 
expect that HQMMs are able to produce more complex time series than HMMs, even 
when using a similar amount of resources. 

It has already been shown that HQMMs encompass HMMs [5]. This means, they are 
able to simulate the same output sequences as HMMs with the same number of internal 
states. However, it is not known how much more efficient HQMMs are in producing 
long-range correlations. To show that HQMMs are indeed more efficient, we recently 
determined the maximum probability for HQMMs and for HMMs to generate words 
of the form 100. ..0001 as a function of the words length. As reported in Ref. [9], we 
found that 1-qubit HQMMs are able to produce stronger correlations than 1-bit HMMs. 
However, a systematic study of the efficiency and the information theoretic and physical 
properties of HQMMs is still outstanding. 
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